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Detailed Action 



Claims 1-13 are pending in this application. 

35 USC § 101 

35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, 
manufacture, or composition of matter, or any new and useful improvement thereof, 
may obtain a patent therefor, subject to the conditions and requirements of this title. 

Claims 1-13 are rejected under 35 U.S.C. 101 because the language of the 
claims raises a question as to whether the claim is directed merely to an abstract idea 
that is not tied to a technological art, environment or machine which would result in a 
practical application producing a concrete, useful, and tangible result to form the basis 
of statutory subject matter under 35 U.S.C. 101. The word/phrases 'computer readable 
medium' and 'method' used in all these claims does not link the abstract concept to the 
technological art, environment or machine. It is suggested the applicant use the phrase 
"computer implemented" in the preamble of independent claim(s) 1 and 9 to describe 
the invention since this would cure the deficiency. 



Claim Rejections - 35 USC § 103 



The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed 
or described as set forth in section 102 of this title, if the differences between the 
subject matter sought to be patented and the prior art are such that the subject matter 
as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall 
not be negatived by the manner in which the invention was made. 

Claims 1-13 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Manjunath and Deng, in view of Manjunath and Ma, in view of Kadtke et al, in view of 
Melen, in view of Wagner, in view of Shu ( Issues for Image/Video Digital Libraries, 
referred to as MD; Texture Features for Browsing and Retrieval of Image Data, referred 
to as MM; U. S. Patent 6278961, referred to as Kadtke; U. S. Patent 5719960, referred 
to as Melen; U. S. Patent 5950180, referred to as Wagner; U. S. Patent 4901360, 
referred to as Shu). 

Claim 1. 

MD teaches (a) a task component configured to perform a plurality of 
classification tasks arranged in [(a]] an established sequential progression of decision 
making (MD, p595, C2: 10-11; Examiner's Note (EN) The first stage, segmentation 



search is the first of a plurality of classification tasks. This is the first part of a system 
decision system software.), 

said established sequential progression of decision making including a plurality of 
classification nodes for assigning class labels to an individual image file of said image 
files of non-textual subject data(MD, p595, C2: 15-20; EN The segmentation part is the 
first step in the established progression of decision making. The next step is the looking 
for local image features which fall under texture, color and shape.), 

at least some of said classification nodes including algorithms for determining 
which of a plurality of alternative next classification nodes Is to be encountered in said 
sequential progression of decision making(MD, p595, C2:41-43; EN In this reference, a 
query pattern request will be used to find a match. This illustrates a plurality of next 
classification nodes to go to.); 

(b) an algorithmic component having access to a storage of available algorithms 
for execution at said classification nodes, said algorithmic component being common to 
said classification nodes and being accessed by each said classification node for 
selecting a specific algorithm for each of said classification tasks(MD, p395 C2:44 
through p596. C1 :6; EN Here the 'access to a storage of available algorithms 1 of 
applicant is equivalent to Visual thesaurus' of MD. The various levels of hierarchy of MD 
is equivalent to 'classification nodes being accessed by each said classification nodes 1 
of applicant.), 

said specific algorithm being configured to execute at least one of content based 
analysis for processing content-based data (MD, p596, C1:11 through p596, C2:5). 

MD does not teach meta-data analysis for processing meta-data. MM teaches meta- 
data analysis for processing meta-data (MM, p841, C1:18-20). It would have been obvious to a 



person having ordinary skill in the art at the time of applicant's invention to modify the method 
of determination of classification of MD with meta-data analysis for processing meta-data by 
MM. Having information about the non-textual file besides the content of the file itself is 
needed as well for classification purposes. 

MD teaches for at least some of said classification nodes said algorithmic 
component is configured to select among alternative stored algorithms that are specific 
to determining assignment of a same said class label (MD, p596, C2:28-31 ; EN After 
the image is segmented into regions by the first algorithm, alternative algorithms such 
as color, shape texture and location are possible used), 

said algorithmic component being further configured to use prior determinations 
at said classification nodes as a basis for selecting among said alternative stored 
algorithms specific to determining assignment of said same class label (MD, p597, 
C1 :16 through C2:4; EN the 'prior determinations' of applicant is equivalent to 'sub- 
objects' in number of video frames in sequence.); 

(c) a sub-algorithmic component for selecting at least one sub-algorithmic routine 
for said specific algorithm having a plurality of sub-algorithm routines, said at least one 
sub-algorithmic routine being selected based on said selecting said algorithm (MD, 
p597, C2:5-16; EN Sub-object information is now generated and using that, sub- 
algorithms of spatial and temporal relations are now used.); and 

(d) a learning component for modifying said arrangement of classification tasks 
according to determinations of frequency patterns In the common assignments of said 
.class labels to individual said image files of non-textual subject data(MD, p596, C2:5-9; 
EN A hybrid neural network is a design that learns from exposure to frequency of 
patterns). 



Claim 2. 

MD teachs a system web-service module for providing Internet access to said 
system decision module (MD, p596 C2:41 through p597 C1:2). 

Claim 3. 

MD teachs a system interface module for providing communications among a 
plurality of system and nonsystem modules, wherein one of said system modules Is 
said system decision module (MD, p596, C2:37-41). 

Claim 4. 

MD teachs non-system modules includes at least one said sub-algorithmic 
routine (MD, p597, C1:5-7; EN Non-system modules such as object tracking scheme 
would have working algorithm(s) to make them functional. 

Claim 5. 

MD and MM do not teach the system interface module further includes data 
components for storing data associated with classifying a plurality of said image files of 
said non-textual subject data and at least one control component for executing said 
sub-algorithmic routines. Kadtke teaches the system wherein said system interface 
module (Kadtke, Figure 2; EN The 'system interface' of applicant enables 
communication between modules. In Figure 3, you can see the communication 
between modules 201, 202, 203, 204 and 205 of Kadtke.) further includes data 
components for storing data associated with classifying a plurality of said image files of 
said non-textual subject data (Kadtke, C2:44-47; EN The 'storing data' of applicant is 



equivalent to 'build and modify a database of features' of Kadtke.) and at least one 
control component for executing said sub-algorithmic routines (Kadtke, Figure 1; EN 
Kadtke shows in control component 105, has three different sub-algorithms to choose 
from.). It would have been obvious to a person having ordinary skill in the art at the 
time of applicant's invention of a system for classifying files of non-textual subject data 
of MD with the system wherein said system interface module further includes data 
components for storing data associated with classifying a plurality of said image files of 
said non-textual subject data and at least one control component for executing said 
sub-algorithmic routines by Kadtke. This allows for ease of communication of new 
modules if added and little effect if removed, having a storage facility with images and 
associated classification for ease of retrieval, and easing system design by having 
multiple sub-algorithms associated to a single node. 

Claim 6. 

MD, MM and Kadtke do not teach a media input/output module for administering 
data associated with classifying said non-textual subject data by reading and writing 
said data among a plurality of modules. Melen teaches a media input/output module for 
administering data associated with classifying said non-textual subject data by reading 
and writing said data among a plurality of modules (Melen, C7:15-18). It would have 
been obvious to a person having ordinary skill in the art at the time of applicant's 
invention of a system for classifying files of non-textual subject data of MD with the 
system further comprising a media input/output module for administering data 
associated with classifying said non-textual subject data by reading and writing said 
data among a plurality of modules by Melen. This allows for faster and more secure 
data input. 



Claim 7. 

MD and MM do not teach the learning component is configured to identify an 
algorithm for each of said classification tasks and at least one sub-algorithmic routine 
for said algorithm. Kadtke teaches the learning component is configured to identify an 
algorithm for each of said classification tasks and at least one sub-algorithmic routine 
for said algorithm (Kadtke, C7:66 through C8:5; EN the learning component of applicant 
is equivalent to 'neural nets' of Kadtke. A neural network can take in data from and 
algorithm and generate the correct sub-algorithm to use for the next classification step. 
It would have been obvious to a person having ordinary skill in the art at the time of 
applicant's invention of a system for classifying files of non-textual subject data of MD 
with teaches the system wherein said learning component is configured to identify an 
algorithm for each of said classification tasks and at least one sub-algorithmic routine 
for said algorithm by Kadtke. Using a neural net for this has the advantages of learning 
and dynamically changing which sub-algorithm to choose from given the inputted data. 

Claim 8. 

MD, MM, Kadtke and Melen do not teach the data capturing device configured to 
capture said content-based data and record said meta-data, said content-based data 
corresponding to content information of a file of said subject data and said meta-data 
corresponding to situational environmental data of said data capturing device during a 
capture of said subject data. Wagner teaches the data capturing device configured to 
capture said content-based data and record said meta-data, said content-based data 
corresponding to content information of a file of said subject data and said meta-data 
corresponding to situational environmental data of said data capturing device during a 



capture of said subject data (Wagner, C4:10-12). It would have been obvious to a 
person having ordinary skill in the art at the time of applicant's invention of a system for 
classifying files of non-textual subject data of MD with the system of claim 1 further 
comprising a data capturing device configured to capture said content-based data and 
record said meta-data, said content-based data corresponding to content information of 
a file of said subject data and said meta-data corresponding to situational environmental 
data of said data capturing device during a capture of said subject data by Wagner. This 
permits input to come from the real world thus having the ability to work in today's 
situation. 

Claim 9. 

MD teaches establishing a sequential progression of decision making, including 
using automated processing techniques to define a dependent arrangement of a 
plurality of task nodes, each said task node in said dependent arrangement being 
associated with a class label for classifying a data file (MD, 595, C2:9-14; EN 
Segmentation is the first step of an established progression of decision making. Region 
based search techniques are the next step with this design. Region based is dependent 
of segmentation and they are within their own separate classes of algorithms), 

MD teaches at least some of said task nodes including algorithms for determining 
which alternative next task node is to be selected in said sequential progression of 
decision making, said task nodes including multi-algorithmic task nodes having a 
plurality of alternative said algorithms for implementing said determination, each said 
multi-algorithmic task node being specific to determining assignment of a particular said 
class label and each said alternative algorithm at said multi-algorithmic task node being 



specific to said particular class label (MD, p595, C2: 15-20; EN As stated above, the 
region based search follows a sequential pattern for decision in classification. In region 
based search there are a number of areas which the method can choose from. Texture, 
color, location are but a few multi-algorithms not within the same class which the task 
node can pick from.). 

MD, MM, Kadtke and Melen do not teach receiving a file of non-textual subject 
data, Wagner teaches receiving a file of non-textual subject data (Wagner, C4:10-12). 
It would have been obvious to a person having ordinary skill in the art at the time of 
applicant's invention of a system for classifying files of non-textual subject data of MD 
with receiving a file of non-textual subject data by Wagner. Using a CCD camera is a 
standard method for an image capturing device. 

MD and MM do not teach progressing said file through said dependent 
arrangement defined in said establishing said sequential progression of decision 
making. Kadtke teaches progressing said file through said dependent arrangement 
defined in said establishing said sequential progression of decision making (Kadtke, 
Figure 2; EN This shows a flow chart that is sequential and containing dependent 
arranged modules). It would have been obvious to a person having ordinary skill in the 
art at the time of applicant's invention of a system for classifying files of non-textual 
subject data of MD with progressing said file through said dependent arrangement 
defined in said establishing said sequential progression of decision making by Kadtke. 
By using a standard consistence approach, this allows for modifications to be made with 
little effect on other modules. 

MD and MM do not teach selecting from among said alternative algorithms at 
said multi-algorithmic task nodes. Kadtke teaches selecting from among said 
alternative algorithms at said multi-algorithmic task nodes ( Kadtke, Figure 1). It would 



have been obvious to a person having ordinary skill in the art at the time of applicants 
invention of a system for classifying files of non-textual subject data of MD with 
selecting from among said alternative algorithms at said multi-algorithmic task nodes by 
Kadtke. This allows for a simpler design when constructing the decision making 
process. 

MD and MM do not teach utilizing an algorithmic component to perform said 
selection, said selection at least partially based on prior determinations at previously 
encountered task nodes in said sequential progression. Kadtke teaches utilizing an 
algorithmic component to perform said selection, said selection at least partially based 
on prior determinations at previously encountered task nodes in said sequential 
progression (Kadtke, C9:25-29). It would have been obvious to a person having 
ordinary skill in the art at the time of applicant's invention of a system for classifying files 
of non-textual subject data of MD with utilizing an algorithmic component to perform 
said selection, said selection at least partially based on prior determinations at 
previously encountered task nodes in said sequential progression by Kadtke. By using 
previous encountered task nodes and statically methods, the performance of the 
method can be improved upon. 

Claim 10. 

MD teaches a learning procedure in which content-based data is extracted from 
each of a plurality of training images and meta-data is identified for each said training 
image (MD, p596, C2:34-35). 



Claim 11. 



MD, MM, Kadtke and Melen do not teach the step of generating a plurality of 
learning classes that are descriptive of said training Images, including using an 
association pattern technique of recognizing and using patterns In assignments of said 
class labels, said step of generating including applying content-based analysis for said 
content-based data and meta-data analysis for said meta-data. Wagner teaches a step 
of generating a plurality of learning classes that are descriptive of said training Images, 
including using an association pattern technique of recognizing and using patterns In 
assignments of said class labels, said step of generating including applying 
content-based analysis for said content-based data and meta-data analysis for said 
meta-data (Wagner, C5:36-39). It would have been obvious to a person having ordinary 
skill in the art at the time of applicant's invention of a system for classifying files of non- 
textual subject data of MD with generating a plurality of learning classes that are 
descriptive of said training Images, including using an association pattern technique of 
recognizing and using patterns In assignments of said class labels, said step of 
generating including applying content-based analysis for said content-based data and 
meta-data analysis for said meta-data by Wagner. EN By using a neural network, 
associating patterns and recognizing said patterns is easily accomplished. 



Claim 12. 

MD, MM, Kadtke and Melen do not teach the step of dynamically modifying said 
sequential progression of decision making, including monitoring said determinations at 
each of said decision nodes and adjusting for detected patterns in said determinations. 
Wagner teaches the step of dynamically modifying said sequential progression of 



decision making, including monitoring said determinations at each of said decision 
nodes and adjusting for detected patterns in said determinations (Wagner, Figure 4; EN 
By using a neural network, dynamically modifying the decision making progress is 
already there in the design.). It would have been obvious to a person having ordinary 
skill in the art at the time of applicant's invention of a system for classifying files of non- 
textual subject data of MD with a step of dynamically modifying said sequential 
progression of decision making, including monitoring said determinations at each of said 
decision nodes and adjusting for detected patterns in said determinations by Wagner. 
Having the ability to dynamically alter the decision making progress, will have the effect 
of a more efficient machine and the ability to change with the current environment. 

Claim 13. 

MD, MM, Kadtke, Melen and Wagner do not teach a step of assigning a semantic 
description to said file of non-textual subject data for one of organizing said file and 
matching a query during a search for said file. Shu teaches a step of assigning a 
semantic description to said file of non-textual subject data for one of organizing said file 
and matching a query during a search for said file (Shu, C14:1-4). It would have been 
obvious to a person having ordinary skill in the art at the time of applicant's invention of 
a system for classifying files of non-textual subject data of MD with a step of assigning a 
semantic description to said file of non-textual subject data for one of organizing said file 
and matching a query during a search for said file by Shu. This will act as another link 
between the user and the file for organizing purposes. 



Conclusion 

The prior art of record and not relied upon is considered pertinent to the 
applicant's disclosure. 

-NeTra-V: Toward an Object_based Video Representation: Yining Deng, B. S. 
Manjunath. 

-Peer Group Filtering and Perceptual Color Image Quantization: Yining Deng, 

Charles Kenney, Michael S. Moore, B. S. Manjunath. 

-Spatio-teporal Relationships and Video Object Extraction:Yining Deng, B. S. 

Manjunath. 

-Content-based Search of video Using Color, Texture and Motion: Yining Deng, 
B. S. Manjunath. 

Claims 1-13 are rejected. 
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